A computational model that can recover an object’s three-dimensional shape from only one of its two-dimensional retinal representations

Figure-ground organization must be given to the model because it has no provisions for establishing figure-ground organization on its own. This means that the model can only come into play after the shape in the 2D image of the 3D shape has be established. To do this, the model is provided with information about which: (i) points form edges in the image, (ii) edges and vertices form contours of faces “out there”, (iii) edges and vertices are symmetric edges and vertices “out there”, and (iv) edges and vertices define volume of the 3D object “out there” (this last requirement is less critical because once points and edges are represented in the 3D space, their convex hull defines volume uniquely). This information must be provided because the a priori constraints used by the model constrain shape. We call our constraints: “symmetry, planarity, maximum compactness and minimum surface.” In our usage, symmetry refers to the mirror-symmetry of the object, planarity refers to the planarity of the contours of the object. The compactness of the object is defined as V/S where V is the object’s volume and S is the object’s surface area. The minimum surface of the object is defined as the minimum of the total surface area. It is important to realize that no depth cues, not even binocular disparity, are used to recover the 3D shape of an object from its 2D retinal representation in our model. Depth is superfluous to its operation. Our maximum compactness and minimum surface constraints are completely novel in the sense that they have never been used in a model designed to recover 3D shape. Symmetry and planarity constraints had been used in models that recover 3D shape before. Maximizing compactness is equivalent to maximizing the volume of an object, but keeping its surface area constant. It is also equivalent to minimizing surface area, but keeping the object’s volume constant. Note that the minimum surface constraint is equivalent to minimizing the object’s thickness. The bottom line is that our model recovers the 3D shape of an object from its 2D retinal shape by selecting a 3D shape that is as compact and, also as thin, as possible, from the infinitely large family of 3D symmetrical shapes that have planar contours consistent with the 2D retinal shape used to recover the 3D object. Another way of saying this is that our recovery of the object’s 3D shape reflects a compromise between our novel maximum compactness and minimum surface constraints. The reader should realize that our model belongs to the class of regularization models designed to solve inverse problems (Poggio et al., 1985).